Evaluation and Understandability of Face Image Quality Assessment

Face image quality assessment (FIQA) has been an area of interest to researchers as a way to improve the face recognition accuracy. By filtering out the low quality images we can reduce various difficulties faced in unconstrained face recognition, such as, failure in face or facial landmark detection or low presence of useful facial information. In last decade or so, researchers have proposed different methods to assess the face image quality, spanning from fusion of quality measures to using learning based methods. Different approaches have their own strength and weaknesses. But, it is hard to perform a comparative assessment of these methods without a database containing wide variety of face quality, a suitable training protocol that can efficiently utilize this large-scale dataset. In this thesis we focus on developing an evaluation platfrom using a large scale face database containing wide ranging face image quality and try to deconstruct the reason behind the predicted scores of learning based face image quality assessment methods. Contributions of this thesis is two-fold. Firstly, (i) a carefully crafted large scale database dedicated entirely to face image quality assessment has been proposed; (ii) a learning to rank based large-scale training protocol is devel- oped. Finally, (iii) a comprehensive study of 15 face image quality assessment methods using 12 different feature types, and relative ranking based label generation schemes, is performed. Evalua- tion results show various insights about the assessment methods which indicate the significance of the proposed database and the training protocol. Secondly, we have seen that in last few years, researchers have tried various learning based approaches to assess the face image quality. Most of these methods offer either a quality bin or a score summary as a measure of the biometric quality of the face image. But, to the best of our knowledge, so far there has not been any investigation on what are the explainable reasons behind the predicted scores. In this thesis, we propose a method to provide a clear and concise understanding of the predicted quality score of a learning based face image quality assessment. It is believed that this approach can be integrated into the FBI’s understandable template and can help in improving the image acquisition process by providing information on what quality factors need to be addressed

Attractive or Aggressive? A Face Recognition and Machine Learning Approach for Estimating Returns to Visual Appearance

A growing literature documents the presence of appearance premia in labor markets. We analyze appearance premia in a high-profile, high-pay setting: head football coaches at bigtime college sports programs. These employees face job tasks involving repeated interpersonal interaction on multiple fronts and also act as the “face” of their program. We estimate the attractiveness of each employee using a neural network approach, a pre-trained Convolutional Neural Network fine tuned for this application. This approach can eliminate biases induced by volunteer evaluators and limited numbers of photos. We also use this approach to estimate the perceived aggressiveness of each employee based on observable facial features. Aggressiveness can be detected from facial characteristics and may be a trait preferred by managers and customers in this market. Results show clear evidence of a salary premium for less attractive employees. No beauty premium exists in this market. We also find evidence of an aggressiveness premium, as well as evidence of higher attendance at games coached by less attractive and more aggressive appearing coaches, supporting customer based preferences for the premia. We also provide a methodological contribution by incorporating face recognition and computer vision analysis to evaluate employee appearance